Evaluation of Mutual information versus Gini index for stable feature selection

The selection of highly discriminatory features has been crucial in aiding further advancements in domains such as biomedical sciences, high-energy physics and e-commerce. Therefore evaluation of the robustness of feature selection methods to small perturbations in the data, known as feature selection stability, is of great importance to people in these respective fields. However, little research has been focused on investigating the stability of feature selection algorithms independently from any learning models. This project address the problem by providing an overview of several established stability measures and reintroduces Pearson’s correlation coefficient as another. The coefficient has then been employed in the empirical evaluation of four commonly used feature selection criteria, Mutual information maximisation, Mutual information feature selection, Gini index and ReliefF. A high overall stability of Mutual information maximisation and Gini index can be observed for small data samples, with a slightly lower stability being seen for ReliefF. All criteria exhibit low stability when applied to high-dimensional datasets, consisting of small number of samples, with Mutual information feature selection performing poorly across all datasets.